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ABSTRACT 



This study represents an attempt at determining if a construct, 
rather than an instructional method, is supported by scientifically-based 
research. The purpose of this study was to examine the credibility of 
evidence-based claims underlying the literature related to Pedagogical 
Content Knowledge (PCK) (Shulman, 1986). Using a hybrid cross- 
validation method, and criteria from the No Child Left Behind Act 
(NCLB, 2001), the National Research Council (NRC, 2002), and the 
Education Sciences Reform Act (ESRA, 2002), the study analyzed and 
evaluated 51 published journal articles containing PCK in the title. 
Findings indicate that PCK can be labeled a “successful program of 
research” and represent the intent of the “scientifically-based research 
standards”; however, no single published article meets all the criteria for 
“scientifically-based research.” Thus, PCK as a construct defined by this 
sample of literature is not based upon scientifically-based research. This 
finding calls into question the application of PCK within the NCATE 
accreditation evaluation criteria and the applications of PCK within 
teacher education. Questions raised and implications for teacher education 
are addressed. Further, recommendations for re-examining both the 
ideology underlying PCK, as well as SBR, is presented. 



INTRODUCTION 

“Given the times, PCK rather quickly and with minimal disturbance 
slipped into teacher educator rhetoric. Today, the concept is almost taken 
for granted as though representing common sense” (Bullough Jr., R.V., 
2001 ). 

“Without methods courses to learn pedagogical content knowledge, 
novices are unlikely to provide quality instruction” (Laczko-Kerr & 
Berliner, 2003, p. 35). 



Since its introduction in the literature base in 1986, the construct known as “pedagogical 
content knowledge,” (Shulman, 1986) otherwise referred to as PCK, has become 
commonplace in education vernacular. So much so, Bullough (2001), acknowledged 
PCK’s new status when he wrote that the concept, “is almost taken for granted as though 
representing common sense” (Bullough Jr., R.V., 2001). Further, by 2003, often times 
the phrase “PCK” no longer needs reference to Shulman and appears in literature as an 
established variable in a causal relationship. For example, Faczko-Kerr & Berliner 
(2003) write, “Without methods courses to learn pedagogical content knowledge, novices 
are unlikely to provide quality instruction” (p. 35). 
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This simple claim, learning PCK is linked to quality instruction; hence, student 
achievement, may or may not be justifiable given the current research-base underlying 
PCK. Further, PCK has entered into a very political arena. In their Spring 2003 
revisions, the National Council for Accreditation of Teacher Education included PCK as 
part of the evaluation criteria for Standard 1 (NCATE, 2002). NCATE lists the first 
standard for assessing potential teacher candidate’s performance as: 

Standard 1: Candidate Knowledge, Skills, and Dispositions 

Candidates preparing to work in schools as teachers or other professional 
school personnel know and demonstrate the content, pedagogical, and 
professional knowledge, skills, and dispositions necessary to help all 
students learn. Assessments indicate that candidates meet professional, 
state, and institutional standards (NCATE, 2002, 
http://www.ncate.org/standard/unit stnds ch2.htm#stndl) . 

Recently, as documented as “Spring 2003 Revisions,” NCATE updated the rubric for 
assessing institutions alignment with Standard 1. Although the phrase “pedagogical 
content knowledge” is not listed in Standard 1 above, the revised rubric includes criteria 
to evaluate institutions effectiveness of “Pedagogical Content Knowledge for Teacher 
Candidates.” More specifically, for the highest rating in the rubric, otherwise known as 
the “target,” the NCATE web-site states the following criteria: 

TARGET 

Teacher candidates reflect a thorough understanding of pedagogical 
content knowledge delineated in professional, state, and institutional 
standards. They have in-depth understanding of the subject matter that 
they plan to teach, allowing them to provide multiple explanations and 
instructional strategies so that all students learn. They present the content 
to students in challenging, clear, and compelling ways and integrate 
appropriately (http://www.ncate.org/standard/unit stnds ch2.htm#stndl) 

(italics added). 

Given that the original document (i.e., Shulman, 1986) delineated between two types of 
knowledge bases, pedagogical knowledge and pedagogical content knowledge, the 
following question arises: Does the “pedagogical” listed in Standard 1 above mean the 
same thing to NCATE as the “pedagogical content knowledge” used in the evaluation 
rubric? The actual answer to this question is not of importance here, the simple fact that 
the phrase “pedagogical content knowledge” has snuck into a very political environment 
before a critical examination of the research base underlying the construct was conducted 
is problematic. 

In short, in only sixteen years, PCK has grown from the first introduction into the 
literature base, to the global label for the ‘what’ teacher education students potentially 
learn in methods courses, to being related to quality instruction, to becoming evaluation 



4 



criteria for NCATE. With the political nature of PCK established, the question arises: is 
the research base underlying PCK sound? 

Along side the development of PCK, in a mere two years, 2001 and 2002, three different 
sets of guidelines were published defining scientific-based research. Although each set 
uses much of the same terminology, each set is unique to a certain degree. 

With the advent of No Child Left Behind Act (NCLB) of 2001, “scientifically-based 
research” (SBR), has also become a popularized phrase in education vernacular. 
Referencing NCLB, Olson & Viadero (2002) provide the following definition of SBR as 
it appears in the 2001 act: 

The term ‘scientific-based research’ means 

(A) research that involves the application of rigorous, systematic, and objective 

procedures to obtain reliable and valid knowledge relevant to education 

activities and programs; and 

(B) research that: 

• employs systematic, empirical methods that draw on observation or 
experiment; 

• involves rigorous data analyses that are adequate to test the stated 
hypotheses and justify the general conclusions drawn; 

• relies on measurements or observational methods that provide reliable and 
valid data across evaluators and observers, across multiple measurements 
and observations, and across studies by the same or different investigators; 

• is evaluated using experimental or quasi-experimental designs in which 
individuals, entities, programs, or activities are assigned to different 
conditions and with appropriate controls to evaluate the effects of the 
condition of interest, with a preference for random- assignment 
experiments, or other designs to the extent that those designs contain 
within-condition or across-condition controls; 

• ensures that experimental studies are presented insufficient detail and 
clarity to allow for replication or, at a minimum, offer the opportunity to 
build systematically on their findings; and 

• has been accepted by a peer-reviewed journal or approved by a panel of 
independent experts through a comparably rigorous, objective, and 
scientific review” (p. 1) 

In response to SBR, the National Research Council presented Scientific research in 
education (NRC, 2002). Given the timeframe of SBR, and NRC’s report, a recent 
Educational Researcher , 31(8), November 2002, devoted an entire themed issue to 
“Scientific Research in Education.” 
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As central participants in crafting the NRC report, Feuer, Towne, and Shavelson (2002) 
were asked to provide the opening article for the ER. In it, they outline the National 
Research Council report’s Principles of Scientific Inquiry. The author’s write: 

Although no universally accepted description of the principles of inquiry 
exists, we argue nonetheless that all scientific endeavors 

• Pose significant questions that can be investigated empirically, 

• Link research to relevant theory, 

• Use methods that permit direct investigation of the questions, 

• Provide a coherent and explicit chain of reasoning, 

• Yield findings that replicate and generalize across studies, and 

• Disclose research data and methods to enable and encourage professional 
scrutiny and critique. 

These principles need to be understood not as an algorithm, checklist, or how-to 
guide but rather as norms of behavior that reflect expectations for how scientific 
research will be conducted. It is very unlikely that any one study would possess 
all of these qualities although a successful program of research is likely to 
embody all of them (p. 7). 

Although SBR and principles of successful research entered the landscape, the Education 
Sciences Reform Act of 2002 also proposed a third set of definitions and/or criteria 
outlining scientifically based research. The act proposes the following “scientifically 
based research standards”: 

(A) The term “scientifically based research standards” means research 

standards that 

• Apply rigorous, systematic, and objective methodology to obtain 
reliable and valid knowledge relevant to education activities and 
programs; and 

• Present findings and make claims that are appropriate to and 
supported by the methods that have been employed. 

(B) The term includes, appropriate to the research being conducted - 

• Employing systematic, empirical methods that draw on observation 
or experiment; 

• Involving data analyses that are adequate to support the general 
findings; 

• Relying on measurements or observational methods that provide 
reliable data; 

• Making claims of causal relationships only in random assignment 
experiments or other designs (to the extent such designs 
substantially eliminate plausible competing explanations for the 
obtained results); 
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• Ensuring that studies and methods are presented in sufficient detail 
and clarity to allow for replication or, at a minimum, to offer the 
opportunity to build systematically on the findings of the research; 

• Obtaining acceptance by a peer-reviewed journal or approval by a 
panel of independent experts thought a comparably rigorous, 
objective, and scientific review; and 

• Using research designs and methods appropriate to the research 
question posed (Education Sciences Reform Act, 2002). 

With the establishment of SBR (NCLB, 2001), principals of successful research 
programs (NRC, 2002), and scientifically based research standards (ESRA, 2002), no 
study was found to date that represented an attempt at examining a construct from within 
the field of Education to determine if the construct met the intent of any or all of these 
research guidelines. 

DATA SOURCES 

The selection of data sources was guided by a purposeful sampling umbrella and used a 
combination of “homogeneous sampling,” and “maximal variation sampling” techniques 
(Creswell, 2002). 

Using the phrase “pedagogical content knowledge” for a keyword search in both the 
ERIC (FirstSearch) and ArticleFirst search engines, over 600 hits occur; further, 349 hits 
list PCK as a subject header to identify the paper. 

Creswell (2002) defines homogeneous sampling as, “purposeful samples individuals or 
sites based upon membership in a subgroup that has defining characteristics” (p. 196). 

To increase the effectiveness of homogeneity sampling, the defining characteristic for the 
subgroup was articles that contain PCK in their title. This sampling technique reduced 
the initial group of 349 down to 83. Further, and due to SBR requiring a peer-review, the 
sample of 83 was narrowed to only 5 1 articles that were published in a recognized peer- 
reviewed journal. 

The maximal variation sampling technique is considered the range of research questions, 
methods, and subject disciplines represented within the data set. Creswell (2002) defines 
maximal variation sampling as, “a purposeful sampling strategy in which the researcher 
samples cases or individuals that differ on some characteristic” (p. 194). The purpose, 
research question, participants, and content field represented by the each of the 
individually published articles are all examples of the maximal variation within the 
sample. 

METHOD/ANALYSIS 

This study combined techniques appropriate for historical analysis and cross-validation to 
create a hybrid historical cross-validation method. In particular, the method was to 
conduct an “internal criticism” (Borg & Gall, 1989, p. 822-4) as a means for cross- 
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validating the literature base. Although cross-validation is typically concerned with re- 
sampling and/or replication, this study attempts to use a more holistic method of cross- 
validating to examine a population of articles. 

The internal criticism “involves evaluating the accuracy and worth of the statements 
contained in a historical document” (p 822). Whereas the task of the historian conducting 
an internal criticism is, “to combine one or more witnesses’ accounts, admittedly 
subjective, and to interpret them (admittedly, also a subjective process) in an attempt to 
discover what actually happened” (p. 823). Thus, the accuracy and worth was 
determined by evaluating the data sources against the criteria for SBR, successful 
program of research and the research standards. 

Given this study, the data sources are actual published articles that which represent the 
historical documents. The witnesses’ accounts, therefore, are the conclusions represented 
within the published articles given the research questions and methods used to garner 
evidence underlying the implications and conclusions. 

More specifically, the methods included after collecting the data sources, each article was 
examined in light of how, if at all, the article met the intent of the criteria outlined by 
SBR, successful program of research or the research standards (See Appendix A for 
sample check- sheet). 

Next, each article was sub-categorized into appropriate data sets. For example, all 
articles were dissected into the following sections: a) purpose; b) research questions; c) 
methods; d) participants; and, e) implications/conclusions (for example see Appendix B). 
Then, each sub-category was evaluated to determine if the sub-category meet the criteria 
for SBR, successful program of research, and/or the scientifically based research 
standards. In other words, I literally analyzed what was in the published text against the 
established criteria. 

Lastly, using the heuristic PCK-3D model (Holder & Cochran, 1998 - see Appendix C), 
categorical analyses were conducted on each data set as a whole to determine the overall 
nature of the data set. For example, all research questions were analyzed and categorized 
to determine which component of PCK the question was addressing in an attempt to 
determine if, or which, component of PCK was being addressed through the study 
research questions. This was done to determine if the successful program of research 
criteria, “Yield findings that replicate and generalize across studies” was being met. In 
other words, did similar studies pose similar research questions relating to similar 
components of PCK? 

RESULTS/CONCLUSIONS 

Descriptive results for the 51 articles published in journals that contain PCK in the title 



are: 
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(a) when examined as a whole, the collection of 5 1 articles meet the 
criteria and can be labeled a “successful program of research” (i.e., 
given all the articles, all the criteria for successful program of research 
have been met); 

(b) that no single article can be labeled “scientific-based research”; 

(c) that they appear in 34 different peer-reviewed Journals; 

(d) they utilize the following method type: 



Type or Method: 


Number (percent of total) 


Qualitative 


34 (66.7%) 


Conceptual/historical analysis 


11 (21.6%) 


Mixed Methods 


2 (3.9%) 


Book Review 


1 (1.9%) 


Construction of Assessment Items 


1 (1.9%) 


Textbook Content Analysis 


1 (1.9%); 



(e) that only 12 of the 51 (i.e., 23.5%) contain clearly stated Research 
Question(s); 

(f) they collectively have utilized 855 participants; and, 

(g) and that they represent the following content domains (listed in order 
from most studies published): Science, Mathematics, Physical 
Education, English, Social Studies, Communication Education, 
Educational Psychology; Business, Visual and Performing Arts, 
Linguistics, Computer Science, and Political Science. 

QUESTIONS RAISED 

Initial analysis and results have prompted many questions. Three of which are: 

1. The phrase PCK is used in a very broad sense with multiple interpretations and 
operational definitions as what the construct represents; thus, the generic defining 
questions arise: What is Pedagogical Content Knowledge? And, how does PCK 
develop? 

2. The collective “Conclusions” and “Implications for Teacher Education” sections 
included in the 5 1 articles propose overwhelming support for more research needed in 
this area and widespread agreement that PCK is fundamental to successful teacher 
education; however, generic implication questions arise such as: How can PCK most 
effectively be utilized by teacher educators? And, what is the connection between 
PCK and classroom student achievement? 

3. This study was confounded by the nature of the data sources selected. Given a 
reasonable attempt was made to collect a sample of studies, only studies that 
contained PCK in their title were chosen. One obvious question arises: Do the 
results of this study, based upon only 5 1 articles, generalize to the population of 
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studies (i.e., over 600) that use PCK as a descriptor phrase in the data sources 
section? 

IMPLICATIONS FOR TEACHER EDUCATION 

The educational significance for teacher education and of this study are twofold: 

Contribution to the development of PCK: This study continues to define the boundaries 
of what PCK was, how it has been examined, where it currently is, and provides possible 
areas for future studies. Thus, as a whole, this study contributes greatly to the 
development of current and future understandings of PCK. 

Contribution to the field: Albeit quietly, PCK has become a very political construct in 
the field of teacher education. Given the NCATE revision of Standard l’s evaluation 
criteria, with only 12 studies actually involving a research question and 0 (zero) studies 
utilizing experimental designs, the results of this cross-validation call into question 
whether PCK is scientifically-based and asks if PCK really is what authors claim it to be. 

CONCLUDING THOUGHTS 

A major contribution of this paper is an immediate call for a re-examination of current 
ideology underlying PCK. Further, this call also asks how the field of Education will 
respond to constructs as popular as PCK that appear as Successful Programs of Research 
yet are not based upon NCLB’s definition of Scientific-Based Research. Lastly, as 
constructs such as PCK enter the political dimension of teacher education accreditation 
and become criteria for programs to be evaluated against, one must ask, is it really 
scientifically based research we are after or is appropriate to make decisions based upon 
only a successful program of research? 

Regarding PCK, perhaps Shulman describes it best when he wrote, “The teacher 
comprehends which aspects of the content will be likely to pose the greatest difficulties 
for the pupil’s understanding. The most crucial to leam is not always the most difficult; 
the most difficult is not always the most crucial” (Shulman, 2004, p. 353). SBR, and in 
particular, experimental or quasi-experimental research are generalizably the ‘rule’ and 
not the exception. However, for a teacher to apply their own PCK, they are contextually 
working with a group or even an individual student - generalizably the ‘exception’ . 

Thus, the question arises: Given this study, and that PCK does not meet the criteria for 
SBR, is PCK the exception to the rule? 
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Appendix A 

Criteria Check Sheet for SBR 

• CRITERIA 1: employs systematic, empirical methods that draw on 
observation or experiment; 

• CRITERIA 2: involves rigorous data analyses that are adequate to test the 
stated hypotheses and justify the general conclusions drawn; 

• CRITERIA 3: relies on measurements or observational methods that 
provide reliable and valid data across evaluators and observers, across 
multiple measurements and observations, and across studies by the same 
or different investigators; 

• CRITERIA 4: is evaluated using experimental or quasi-experimental 
designs in which individuals, entities, programs, or activities are assigned 
to different conditions and with appropriate controls to evaluate the effects 
of the condition of interest, with a preference for random- assignment 
experiments, or other designs to the extent that those designs contain 
within-condition or across-condition controls; 

• CRITERIA 5: ensures that experimental studies are presented insufficient 
detail and clarity to allow for replication or, at a minimum, offer the 
opportunity to build systematically on their findings; and 

• CRITERIA 6: has been accepted by a peer-reviewed journal or approved 
by a panel of independent experts through a comparably rigorous, 
objective, and scientific review” (p. 1) 
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Appendix B 



PCK RESEARCH QUESTION(S) CHART 



Article 

Identification 

# 


Stated Research Question 


2 


What do expert teachers know about their subject matter that novice 
teachers do not (p. 59)? 


4 


1. What do teachers know about the distinctions between different 
addition and subtraction problem types? 

2. What do teachers know about the strategies that children use to 
solve different problems? 

3. How successful are teachers in predicting their own students’ 
success in solving different types of problems and in identifying 
the strategies used by children to solve problems of different 
types? 

4. What is the relation between different measures of teachers’ 
pedagogical content knowledge and their students’ achievement 
(p. 389)? 


5 


1. If pedagogical content knowledge is an important component of 
the knowledge base of teaching, does professional education, in 
fact, transmit this area of professional knowledge? 

2. How can strong subject-specific teacher preparation coursework 
influence how beginning teachers develop pedagogical content 
knowledge? 

3. What happens when people enter teaching without professional 
preparation? 

4. Does strong subject matter knowledge alone provide the 
pedagogical understanding of a subject necessary for teaching (p. 
25)? 


15 


How do teachers develop pedagogical thinking processes (p. 263)? 


16 


Two questions guided the research: (a) what knowledge did the 
preservice teachers describe as salient?; and (b) how did this knowledge 
develop (p. 70)? 


23 


This study addresses the question, What are the essential features of the 
modem concept of function (p. 95)? 


34 


In this sense, important questions arise, such as: How do professors 
transcend their status from “subject matter knowers” to “subject matter 
teachers?” (Berliner, 1986, pp. 9-10). How do professors structure and 
implement generic PCK (p. 296)? 


54 


(How) do the characteristics of the lesson preparation method stimulate 
pre- service teachers to show their (ability to develop) pedagogical 
content knowledge (p. 15)? 


61 


For example, what do preservice teachers believe about how scientists 
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work or how learners come to understand scientific ideas? What do we 
know about their backgrounds in learning science? What do they bring 
in their conceptions of learning to teach science? How would 
knowledge of our students’ entering beliefs and knowledge be helpful 
for science teacher educators (p. 30)? 


68 


1. How much (and what kind of) content knowledge do teachers 
need in order to have a rich discussion about hard pedagogical 
content questions such as: what makes this particular science 
easy or hard for students to learn? And what are successful ways 
of making challenging science ideas comprehensible to young 
learners? 

2. To what extent does an integrated teaching case contribute to the 
content and richness of teachers’ discussions? To what extent 
does an integrated case influence how teachers talk about science 
and the pedagogical decisions related to teaching that science (p. 
268)? 


74 


1. What do preservice teachers learn about students’ difficulties 
with slope, and how is this knowledge reflected in their lesson 
plans and in their teaching? 

2. What do preservice teachers learn about various representations 
for teaching slope, and how is this knowledge reflected in their 
lesson plans and in their teaching? Specifically, how do they use 
real-world representations - physical and functional situations 
involving slope (p. 210)? 


76 


What exactly is the significance of the quality of prospective teachers’ 
understanding of subject matter? What effect, if any, does teachers’ 
initial knowledge state (SMK) have on the quality of their instructional 
explanations? Or, put another way, what is the consequence of 
prospective teachers’ understanding and beliefs for their subsequent 
pedagogy (PCK) (p. 52)? 
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Appendix C 

Pedagogical Content Knowing - The Third Dimension (PCKg — 3D) 




KNOWING SELF AS TEACHER 

• Educational Philosophy 

• Professional Development - Reform or Conform? 

• Philosophy of Life 


KNOWING CONTENT 

• Substance (ideas) / Structure (representation) / 
Syntax (construction & verification) 

• Creating new/verifying old 

• Inter-conceivability - across/within 


KNOWING PEDAGOGY 

• Teacher/Student Relationship 

• Instructional Strategies 

• Assessment/Motivation/Learning-Relationship 

• PCK 


KNOWING ENVIRONMENTS 

• Learning Community - School/Classroom 

• Social/Political/Cultural 

• Relationship b/w policy and implementation 


KNOWING STUDENTS 

• Individual Differences 

• Prior Conceptions about Content/School/etc. 

• Needs - school or go hunting? 









